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Sir: 



In response to the Final Office Action mailed June 4, 2004, reconsideration 
and allowance of the present application are respectfully requested. 
Claims 1-20 remain pending in the application. 

On page 2 of the Office Action, claims 1-7, 1 1-12 and 16-19 are rejected under 
35 U.S.C. §1 02(e) as being anticipated by U.S. Patent No. 6,240,423 (Hirata et al). 
This rejection is traversed, because the Hirata patent is directed to a query based 
image matching system, wherein images are displayed only a single time per query. 
Using this method, the results of a database query are displayed based upon results of 
a combined region based image matching and a boundary based image matching. In 
contrast, exemplary embodiments of the present invention are not directed to query 
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based matching, but rather, to using a refined distance metric between each data file to 
redisplay a portion of displayed images.. This is a significant distinction, and is 
reflected in Applicants 1 independent claims 1 and 17. 

The Examiner asserts in numbered paragraph 3 of the Office Action with respect 
to claim 1 : 

Hirata teaches a method of visualizing and retrieving a data file 
comprising: 

Displaying a plurality of images representing corresponding data files on 
a display device using a first distance metric (Calculating a first similarity 
between a query image and images in the database using a region-based 
matching to produce a first set of similar images. Images being retrieved 
in order based upon the distance between the query image and the result 
candidate images, column 6) between each data file (e.g., Based on the 
mutual similarities between the images, retrieval results of the candidate 
images are sorted. The images which are similar to each other are 
assumed to be under one group and are re-ordered based on the 
similarity and users can specify the forms of the output for display; figures 
1-11; column 6-9); 

Redisplaying a portion of the images on the display device using a refined 
distance metric (e.g., Hirata teaches using a distance metric such as a 
second similarity to retrieve images based on the set of similar images 
retrieved using a first similarity. The second similarity is now a "refined" 
distance metric because it refines the search for images. The second 
similarity of Hirata thus meets the claim limitation of a "refined distance 
metric." Here, a refined set of similar images is produced from the first 
set of similar images produced using a first distance metric, figures. 9,11 
and 12; column 6-9); and 

Performing at least one of retrieving, marking and selecting at least one 
desired data file (Hirata teaches retrieving and selecting at least one 
desired data file; see, figures 1-11; column 6-9). 



Later, on page 5 of the Office Action, in numbered paragraph 4, the Examiner 



rejects independent claim 17 based on the Hirata patent. In this portion of the Office 
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Action, the Examiner asserts that the Hirata patent discloses clustering image data 
files using a distance metric such as a first similarity, with reference to Figures 1-1 1 
and at column 6-9. The Examiner also asserts that the Hirata patent discloses: 



Interactively selecting, by a user, a portion of the images (e.g., a user 
specify [ies] the regions of the images; e.g., the region division of figure 
3; figures 1-11; column 6-9); 

Redisplaying the portion of the images in real time on the display 
device using a redefined distance metric (e.g., Hirata teaches using a 
distance metric such as a second similarity to retrieve images based 
on the set of similar images retrieved using a first similarity. The 
second similarity is now a "refined" distance metric because it refines 
the search for images. The second similarity of Hirata thus meets the 
claim limitation of "a refined distance metric." Here, a refined set of 
similar images is produced from the first set of similar images 
produced using a first distance metric. Re-order based on the 
similarity among the candidate images of figure 9 or 2 nd stage image 
matching based on boundary of figure 11 and grouping and re-ordering 
is based on the similarity among candidates using the refined distance 
metric of figure 11; figures 1-11; column 4); and 

Retrieving a desired data file (retrieved data file is displayed on a 
display device; figures 1-11; column 4). 

On pages 8-10 of the Office Action, claims 8 and 13-14 are rejected as being 
unpatentable over the Hirata patent in further view of U.S. Patent No. 6,584,221 
(Moghaddam). On pages 10-1 1 of the Office Action, claims 9-10 and 20 are 
rejected as being unpatentable over the Hirata patent in view of U.S. Patent No. 
6,121,969 (Jain). On pages 11-12 of the Office Action, claim 15 is rejected as being 
unpatentable over the Hirata patent in view of U.S. Patent No. 5,528,259 (Bates). 

The foregoing rejections are respectfully traversed, as the Hirata patent, 
considered alone, or in combination with the various secondary references relied 
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upon by the Examiner, fails to teach or suggest all features set forth in Applicants' 
independent claims 1 and 17. 

The present application discloses a multi-media database and 
classification system, which can, for example, provide for automatic classification 
and retrieval of multimedia files based on features of the multimedia files. Figure 1 
shows an exemplary method for visualizing and retrieving data files in accordance 
with an exemplary embodiment of the present invention. In step 110, a plurality of 
images representing data files on a display device are displayed using a first (i.e., 
coarse) distance metric. The distance metric represents a distance between each 
data file. In step 120, a portion of the images can be redisplayed on the display 
device using a refined distance metric. As described on specification page 6 
beginning with line 4, the distance metric can be refined at each of plural redisplays 
until a desired data file is found or a maximum refined distance metric is reached. In 
step 140, a desired data file can be retrieved, and/or can be marked or selected. 

Figures 2A-2E show a graphical representation of displays generated in 
accordance with exemplary embodiments, where each portion of the images 
redisplayed can be graphically selected by a user. Figure 2A is a screen capture of 
an exemplary display showing a first level of images representing data files on a 
two-dimensional display. A first, coarse, distance metric can be calculated, and 
allows the user to receive useful information about the organization of images on the 
display. Based on perceived properties of a desired image, a user can select an 
area 202 of the screen where a desired image most likely resides. Figure 2B shows 
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a portion of images redisplayed as selected by area 202. Here the distance metric 
has been recalculated using more of the image information than was used in the first 
distance calculation. 

The redisplay process can be repeated, as represented by Figures 2C and 
2D. Distance metrics between images are again recalculated using more image 
feature data than previously used with respect to Figure 2B. Following the redisplay 
based on recalculated distance metrics, a desired data file (e.g., image 240) is 
identifiable and can be selected and retrieved as shown in Figure 2E. Figure 3 
shows another exemplary method which can be implemented in accordance with the 
present invention, and Figure 4 shows a flowchart for performing a coarse to fine 
distance calculation. 

The foregoing features are broadly encompassed by independent claims 1 
and 17. In contrast to using query based image retrieval, exemplary embodiments 
of the present invention display images using a first distance metric between each 
data file, and provide for redisplaying a portion of the images using a refined 
distance metric. The claim 1 method includes a step of displaying a plurality of 
images representing corresponding data files on a display device using a first 
distance metric between each data file. Claim 1 also recites redisplaying a 
portion of the images on the display device using a refined distance metric. Claim 
17 is directed to a method of interactively retrieving a data file from a set of data files 
in real time, and includes features similar to those mentioned with respect to claim 1 . 

The foregoing features are neither taught nor suggested by the Hirata patent 
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considered individually or in combination with the various secondary documents 
relied upon by the Examiner. 

In contrast to the presently claimed invention, the Hirata patent is directed to 
a method for querying a database of images and displaying the results in a manner 
as illustrated, for example, in Figure 10B of the Hirata patent. The results of the 
query are based upon a combination of region based image matching and boundary 
based image matching as described, for example, at column 2, lines 48-65. In 
Hirata's query-based system, an initial query is submitted and a set of results is 
clustered. The clustering of the search results is performed using a distance 
between each file and a distance to the query, as discussed at column 6, lines 37- 
41. In all cases, clustering involves, at least in part, using the distance to a query 
(see Hirata's Figures 8, 9 and 10). The Hirata patent is not directed to redisplaying 
a portion of displayed images using a refined distance metric. In contrast to 
Applicants 1 claimed use of a refined distance metric to redisplay a portion of 
displayed images, Hirata teaches applying a new distance metric to all images 
during each image matching operation. 

Thus, in contrast to the method disclosed and recited in Applicants' 
independent claims 1 and 17, the Hirata patent fails to teach or suggest using a 
refined distance metric to redisplay a portion of displayed images. 

In addition, the Hirata patent fails to teach or suggest the claim 17 feature of 
interactive user selection for selecting a subset of images to be displayed. Rather, 
the query based system of the Hirata patent groups all images, and only creates 
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new groupings based on a new query. The Hirata patent does not teach or suggest 
applying a refined distance metric in a second stage of image matching to redisplay 
a portion of displayed images. 

In rejecting claim 17 on page 6 of the Office Action, the Examiner refers to 
the region division in Figure 3 of the Hirata patent as illustrating user specified 
regions. However, Hirata does not teach or suggest that a refined distance metric, 
as presently claimed, is used to redisplay the images of any defined region (see 
Hirata at column 4, line 10 to column 5, line 13). As such, the Hirata patent fails to 
teach or suggest the user interactive selection feature of Applicant's claim 17. 

Thus, the Hirata patent fails to teach or suggest the features recited in 
Applicants' independent claims 1 and 17. 

The remaining documents relied upon by the Examiner; namely, the patents 
to Moghaddam, Jain and Bates, fail to overcome the deficiencies of the Hirata 
patent. As such, independent claims 1 and 17 are allowable. All of the other claims 
depend from independent claims 1 and 17 and recite additional advantageous 
features which further distinguish over the documents relied upon by the Examiner. 
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All rejections and objections raised in the Office Action having been 
addressed, it is respectfully submitted that the present application is in condition for 
allowance and a Notice of Allowance is respectfully solicited. 

Respectfully submitted, 

Burns, Doane, Swecker & Mathis, L.L.P. 



Date: September 7. 204 By: 
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